Lecture Notes 13 : EM algorithm

In statistics, an expectationmaximization (EM) algorithm is an iterative method for finding maximum likelihood or maximum a posteriori (MAP) estimates of parameters in statistical models, where the model depends on unobserved latent variables. The EM iteration alternates between performing an expectation (E) step, which creates a function for the expectation of the log-likelihood evaluated using the current estimate for the parameters, and a maximization (M) step, which computes parameters maximizing the expected loglikelihood found on the E step. These parameter-estimates are then used to determine the distribution of the latent variables in the next E step. Given a statistical model consisting of a set X of observed data, a set of latent data or missing values Z and a vector of unknown parameters θ. Let Y = (X,Z), called complete data. The maximum likelihood estimate (MLE) of the unknown parameters is determined by the marginal likelihood of the observed data